Method and device for processing video

ABSTRACT

The present disclosure provides a method and device for processing a video. The method includes: determining a special effect video frame of a video, where a target feature area of the special effect video frame includes a preset special effect map; and modifying a display effect of the special effect map upon determining that a shielded area exists in the target feature area.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based on and claims priorities under 35 U.S.C 119 toChinese Patent Application No. 201910590062.5, filed on Jul. 2, 2019,and Chinese Paten Application No. 201910843968.3, filed on Sep. 6, 2019,in the China National Intellectual Property Administration. The entiredisclosure of the above applications is incorporated herein byreference.

FIELD

The embodiments of the present disclosure relate to the field ofcomputers, and particularly relate to a method and device for processinga video.

BACKGROUND

Recently, with the continuous development of the short video industryand the continuous improvement of a real-time render technology, a usermay add some visual special effects to a picture in real time whileshooting a short video to increase the interestingness of the shortvideo. For example, when shooting a portrait picture short video, theuser may add some special effects of makeup materials to a figure in thepicture to improve the attractiveness of the figure.

SUMMARY

The embodiments of the present disclosure provide a method and devicefor processing a video.

In a first aspect, the embodiment of the present disclosure provides amethod for processing a video. The method includes: determining aspecial effect video frame of the video, where a target feature area ofthe special effect video frame comprises a preset special effect map;and modifying a display effect of the special effect map upondetermining that a shielded area exists in the target feature area.

In some embodiments, the modifying the display effect of the specialeffect map includes: extracting a mask corresponding to a special effectvideo frame; determining a weight value of the special effect videoframe based on topological relation information corresponding to thespecial effect video frame, the mask and a preset legal key point upondetermining that the shielded area exists in the target feature area ofthe mask; generating a replacing video frame according to the specialeffect video frame and the weight value; and replacing the specialeffect video frame with the replacing video frame.

In some embodiments, the determining the weight value of the specialeffect video frame according to topological relation informationcorresponding to the special effect video frame, a mask and a presetlegal key point includes: mapping the mask to an empty graph layer witha preset size according to the topological relation information, themask and the preset legal key point; and obtaining the weight valuebased on channel information of the empty graph layer after settingtransparency of the empty graph layer to a preset degree.

In some embodiments, before mapping the mask to the empty graph layerwith the preset size according to topological relation information, themask and the preset legal key point, the method further includes:obtaining a legal image with the preset size, where the legal imageincludes a target feature area; and extracting the preset legal keypoint from the target feature area.

In some embodiments, after obtaining the weight value based on channelinformation of the empty graph layer and by setting transparency of theempty graph layer to a preset degree, the method further includes:extracting a pixel value of each pixel in the empty graph layer;determining a swipe area value according to the pixel value of eachpixel; and replacing the special effect map with another special effectmap in response to the swipe area value being greater than or equal to afirst preset threshold value.

In some embodiments, the determining the swipe area value includes:determining a plurality of target pixels whose pixel values are greaterthan or equal to a second preset threshold value from all the pixels ofthe empty graph layer; and taking a ratio of a sum of the pixel valuesof the plurality of target pixels to the number of the target pixels asthe swipe area value.

In some embodiments, the extracting the pixel value of each pixel in theempty graph layer includes: sending the empty graph layer to a centralprocessing unit through a graphics processor; and extracting the pixelvalue of each pixel in the empty graph layer through the centralprocessing unit.

In some embodiments, the method further includes: determining the weightvalue of the special effect video frame based on the topologicalrelation information, the mask and the preset legal key point upondetermining that a non-mask area exists in the target feature area ofthe mask.

In some embodiments, the method further includes: extracting a pluralityof video frames of the video based on a preset time period; collecting aplurality of target key points of target feature areas in the videoframes; and determining the topological relation information based onrelative position vector information among the plurality of target keypoints.

In some embodiments, the method further includes: increasing a size ofthe mask based on a preset multiple; and fuzzing the mask.

In a second aspect, the embodiment of the present disclosure provides adevice for processing a video. The device includes a memory and at leastone processor, where the at least one processor is configured to readand execute instructions stored in the memory to: determine a specialeffect video frame of a video, where a target feature area of thespecial effect video frame comprises a preset special effect map; andmodify a display effect of the special effect map upon determining thata shielded area exists in the target feature area.

In some embodiments, the at least one processor is further configured toread and execute instructions stored in the memory to: extract a maskcorresponding to the special effect video frame; determine a weightvalue of the special effect video frame based on topological relationinformation corresponding to the special effect video frame, the maskand a preset legal key point upon determining that the shielded areaexists in the target feature area of the mask; generate a replacingvideo frame based on the special effect video frame and the weightvalue; and replace the special effect video frame with the replacingvideo frame.

In some embodiments, the at least one processor is further configured toread and execute instructions stored in the memory to: map the mask toan empty graph layer with a preset size based on the topologicalrelation information, the mask, and the preset legal key point; andobtain the weight value based on channel information of the empty graphlayer after setting transparency of the empty graph layer to a presetdegree.

In some embodiments, the at least one processor is further configured toread and execute instructions stored in the memory to: obtain a legalimage with the preset size, where the legal image includes the targetfeature area; and extract the preset legal key point from the targetfeature area.

In some embodiments, the at least one processor is further configured toread and execute instructions stored in the memory to: extract a pixelvalue of each pixel in an empty graph layer; determine a swipe areavalue according to the pixel value of each pixel; and replace thespecial effect map with another special effect map in response to theswipe area value being greater than or equal to a first preset thresholdvalue.

In some embodiments, the at least one processor is further configured toread and execute instructions stored in the memory to: determine aplurality of target pixels whose pixel values are greater than or equalto a second preset threshold value from all the pixels of the emptygraph layer; and take a ratio of a sum of the pixel values of theplurality of target pixels to the number of the target pixels as theswipe area value.

In some embodiments, the at least one processor is further configured toread and execute instructions stored in the memory to: send the emptygraph layer to a central processing unit through a graphics processor;and extract a pixel value of each pixel in the empty graph layer throughthe central processing unit.

In some embodiments, the at least one processor is further configured toread and execute instructions stored in the memory to: determine theweight value of the special effect video frame based on the topologicalrelation information, the mask and the preset legal key point upondetermining that a non-mask area exists in the target feature area ofthe mask.

In some embodiments, the at least one processor is further configured toread and execute instructions stored in the memory to: extract aplurality of video frames of the video based on a preset time period;collect a plurality of target key points of target feature areas in thevideo frames; and determine the topological relation information basedon relative position vector information among the plurality of targetkey points.

In some embodiments, the at least one processor is further configured toread and execute instructions stored in the memory to: increase a sizeof a mask according to a preset multiple; and fuzz the mask.

In a third aspect, the embodiment of the present disclosure provides anon-transitory storage medium. A computer program is stored on thestorage medium, and when the computer program is executed by aprocessor, the method for processing a video according to the firstaspect is implemented.

In a fourth aspect, the embodiment of the present disclosure furtherprovides an application program, and when the application program isexecuted by a processor of an electronic device, operations of themethod for processing a video provided in the present disclosure areimplemented.

BRIEF DESCRIPTION OF THE DRAWINGS

By reading detailed description of preferred embodiments below, variousother advantages and benefits will become clear to those of ordinaryskills in the art. The drawings are only configured to show purposes ofthe preferred embodiments, and are not considered as limitations to thepresent disclosure. Furthermore, same reference symbols are configuredto denote the same parts throughout the drawings.

FIG. 1 is a flowchart of operations of a method for processing a videoprovided by the embodiments of the present disclosure.

FIG. 2 is a mask provided by the embodiments of the present disclosure.

FIG. 3 is a flowchart of operations of another method for processing avideo provided by the embodiments of the present disclosure.

FIG. 4 is a block diagram of a device for processing a video provided bythe embodiments of the present disclosure.

FIG. 5 is a logic block diagram of an electronic device according to anembodiment of the present disclosure.

FIG. 6 is a logic block diagram of an electronic device according toanother embodiment of the present disclosure.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Hereinafter, exemplary embodiments of the present disclosure will bedescribed in more detail with reference to drawings. Although thedrawings show the exemplary embodiments of the present disclosure, itshould be understood that the present disclosure may be implemented invarious forms and should not be limited by the embodiments set forthherein. Rather, the embodiments are provided to enable a more thoroughunderstanding of the present disclosure and to fully convey the scope ofthe present disclosure to those skilled in the art.

FIG. 1 is a flowchart of operations of a method for processing a videoprovided by the embodiments of the present disclosure. As shown in FIG.1, the method includes the following steps.

Step 101, a special effect video frame set with a preset special effectmap in a video is obtained.

In the embodiments of the present disclosure, a short video applicationof a mobile terminal can call a camera of the mobile terminal to shoot avideo. In the short video application, a large number of special effectmaps are preset. A user may select a corresponding special effect map,and the corresponding special effect map is attached to a video pictureto increase attractiveness of the video picture.

In some embodiments, the special effect map is rendered to an emptygraph layer corresponding to a video frame through a real-time graphicsrendering technology, to obtain the special effect video frame.

For example, for a portrait video picture, assuming that the userdetermines an eye makeup special effect map A and a lip makeup specialeffect map B, the user may render the eye makeup special effect map A toa graph layer a of the video frame in the video frame of a video andrender the lip makeup special effect map B to a graph layer b of thevideo frame, thereby obtaining a special effect video frame (A+a) and aspecial effect video frame (B+b).

Step 102, a mask corresponding to a target feature area is extractedfrom the special effect video frame.

In the embodiments of the present disclosure, determination of the maskis a processing process that a processed image is (wholly or partially)shielded through a selected image, figure or object to control an imageprocessing area. The mask is configured to extract a region of interestin an image to be processed, namely, a pre-made region of interest maskis multiplied by the image to be processed to obtain an image of theregion of interest, where an image value in the region of interestremains unchanged, and image values outside the region are all 0. Themask further has a shielding effect. The mask is configured to shieldcertain areas on the image so as to prevent the certain areas fromparticipating in processing or calculation of processing parameters, oronly process or count the shielded areas. The mask may further achievestructural feature extraction, namely, a structural feature, similar tothe mask, in the image is detected and extracted through a similarityvariable method or an image matching method.

In some embodiments, in the special effect video frame, the maskcorresponding to the target feature area may be extracted throughrelated codes of mask extraction in an Open Source Computer VisionLibrary (opencv), and the mask corresponding to the target feature areamay be output and obtained by inputting a special effect video frameimage into the opencv and dividing the target feature area. For portraitprocessing in the video, the target feature area may be a face area.

FIG. 2 shows a mask provided by the embodiments of the presentdisclosure. FIG. 2 shows a mask 10 obtained by extracting a face area 20of a portrait image, and a part of the face area is shielded with afinger when a user shoots the portrait image, so that a correspondinghand shielded area 30 exists in the mask 10. In the mask 10, an area,without the hand shielded area 30, in the face area 20 may serve as amask area, so that interference in the mask area by processing of amakeup special effect in the face area 20 can be avoided in the laterperiod.

Step 103, if a shielded area exists in the target feature area of themask, a weight value of the special effect video frame is determinedaccording to topological relation information corresponding to thespecial effect video frame, the mask and a preset legal key point.

In this step, referring to FIG. 2, if it is detected that the shieldedarea 30 exists in the mask 10, the weight value of the special effectvideo frame may be determined according to the topological relationinformation corresponding to the special effect video frame, the mask,and the preset legal key point.

In the scene of processing of a portrait, the special effect video framemay be a face image, and acquisition of the topological relationinformation corresponding to the face image may include: selecting aplurality of key points in the face image through a face detectiontechnology, and further determining a mesh topological connectionrelation among an expansion point and the plurality of key points basedon the plurality of key points so as to obtain the topological relationinformation corresponding to the special effect video frame.

In addition, determination of the legal key points may include:selecting a standard face image, selecting a plurality of standard keypoints in the standard face image through the face detection technology,and taking the plurality of standard key points as the legal key points.

In the embodiments of the present disclosure, determining the weightvalue of the special effect video frame according to the topologicalrelation information corresponding to the special effect video frame,the mask and the preset legal key point may include: mapping the mask toan empty graph layer with a preset size according to the legal key pointand a mapping relation expressed by the topological relationinformation, where after corresponding transparency is set for the emptygraph layer, the empty graph layer records all rendering results of arendering operation on a special effect map in the special effect videoframe before the current moment, and the results may be reflected bychannel information of the empty graph layer. Therefore, the weightvalue of the special effect video frame may be further determinedthrough the channel information of the empty graph layer.

Step 104, a corresponding replacing video frame is generated accordingto the special effect video frame and the weight value.

In this step, if there is only one special effect video frame, after thespecial effect video frame and the weight value of the special effectvideo frame are determined, related parameters of the special effect mapin the special effect video frame may be multiplied by the weight valueto obtain a modified replacement video frame, where compared with theoriginal special effect video frame, special effect display intensity ofthe special effect map of the replacement video frame is changed.According to different weight values, changes of the special effectdisplay intensity include but are not limited to enhancement, reduction,complete erasure and the like.

If there are a plurality of special effect video frames, after thespecial effect video frames and weight values of the special effectvideo frames are determined, the plurality of special effect videoframes may be subjected to weighted mix to obtain a mixed replacementvideo frame. The mixed replacement video frame has a plurality ofspecial effect maps, and the different special effect maps havedifferent display effects in the replacement video frame according tothe corresponding different weight values.

Step 105, the corresponding special effect video frame in the video isreplaced with the replacement video frame.

In this step, after the corresponding special effect video frame in thevideo is replaced with the replacement video frame, in a process ofplaying the video, the special effect display intensity of thecorresponding special effect map in the video is changed under an effectof the weight value so as to achieve the purpose of modifying thespecial effect display intensity of the special effect map.

In summary, by means of a method for processing a video provided by theembodiments of the present disclosure, the special effect video frameset with the preset special effect map in the video may be obtained. Inthe special effect video frame, the mask corresponding to the targetfeature area is extracted. If the shielded area exists in the targetfeature area of the mask, the weight value of the special effect videoframe is determined according to the topological relation informationcorresponding to the special effect video frame, the mask and the presetlegal key point. According to the special effect video frame and theweight value, the corresponding replacement video frame is generated.The replacement video frame replaces the corresponding special effectvideo frame in the video. In the present disclosure, through the maskwith the shielded area, the topological relation informationcorresponding to the special effect video frame and the legal key point,the weight value of a correction degree of the special effect map isdetermined. When shooting the video, the user may shield an area withthe special effect map through a body or other objects to achievereal-time special effect display intensity adjustment of the specialeffect map in the area, and during an entire adjustment process, theuser only needs to shield a picture instead of setting the relatedparameters, so that a process of the special effect display intensityadjustment is simplified, and smoothness of the special effect displayintensity adjustment is improved.

FIG. 3 is a flowchart of steps of another method for processing a videoprovided by the embodiments of the present disclosure. As shown in FIG.3, the method may include the following steps.

Step 201, a special effect video frame set with a preset special effectmap in a video is obtained.

Implementation of this step is similar to implementation of step 101above, so the implementation of this step will not be described indetail in the embodiments of the present disclosure.

Step 202, in the special effect video frame, a mask corresponding to atarget feature area is extracted.

Implementation of this step is similar to implementation of step 102above, so the implementation of this step will not be described indetail in the embodiments of the present disclosure.

Step 203, a size of the mask is increased according to a presetmultiple.

In the embodiments of the present disclosure, since a picture in thevideo is displayed dynamically, a position of the target feature area inthe picture will further change continuously along with dynamic changesof the video picture. The mask may be unable to cover the target featurearea, resulting in incomplete correction of the special effect map.

Therefore, in this step, the size of the mask may be enlarged by thepreset multiple, so that the mask can always completely cover the targetfeature area during the dynamic changes of the video picture.

For example, in the video with a portrait picture, a person often moveshis/her head, such as turning the head and nodding. After the size ofthe mask corresponding to a face area is enlarged, it may be ensuredthat when the head is turned in the picture, the face area can becompletely covered with the mask, so that mistreatment of the specialeffect map is avoided.

Step 204, fuzzy treatment is performed on the mask.

In the embodiments of the present disclosure, after the size of the maskis enlarged, fuzzy treatment may be further performed on the mask.Specifically, a linear interpolation method may be adopted to performthe fuzzy treatment, wherein the fuzzy treatment of the mask may furtherspecifically be feathering treatment on an edge of the mask. The purposeof performing the fuzzy treatment on the mask is to enhance softness ofa processing effect of the special effect map, and to avoid a too sharpprocessing effect of the special effect map.

Step 205, a plurality of video frames in the video are extractedaccording to a preset time period.

In this step, the video frame may be extracted from the beginning of thevideo every preset time period until the end of the video, and finallythe plurality of video frames are obtained. In addition, video frameextraction may further be carried out in the video in a random orderaccording to the preset time period to obtain the plurality of videoframes, which is not limited in the embodiments of the presentdisclosure.

Step 206, a plurality of target key points of the target feature area inthe video frame are collected.

In the embodiments of the present disclosure, the plurality of targetkey points of the target feature area in the video frame may be selectedthrough a key point detection technology.

For example, for a portrait processing scene, the plurality of keypoints in a face image may be selected through a face detectiontechnology, where the key points are often location points of a facialfeature outline in a portrait.

Step 207, relative position vector information among the plurality oftarget key points is taken as topological relation information.

In this step, on the basis of the plurality of key points, an expansionpoint connected to the key point and the relative position vectorinformation among the plurality of key points are further determined, sothat topological relation information corresponding to the specialeffect video frame is obtained, wherein the expansion point and therelative position vector information among the plurality of key pointsmay be selected by a graphic designer.

Step 208, if a shielded area exists in the target feature area of themask, a weight value of the special effect video frame is determinedaccording to the topological relation information corresponding to thespecial effect video frame, the mask and a preset legal key point.

Implementation of this step is similar to implementation of step 103above, so the implementation of this step will not be described indetail in the embodiments of the present disclosure.

In some embodiments, step 208 may include sub-step 2081.

Sub-step 2081, if a non-mask area exists in the target feature area ofthe mask, the weight value of the special effect video frame isdetermined according to the topological relation informationcorresponding to the special effect video frame, the mask and the presetlegal key point.

In the embodiments of the present disclosure, if there is an area with apixel value 0 in the target feature area of the mask, the area with thepixel value 0 may be determined as the non-mask area. The non-mask areais configured to reflect the shielded area. In the mask, a backgroundarea outside the non-mask area and the target feature area may betransparent. A mask area is an area of interest, and a pixel value of animage in the mask area remains non-zero and unchanged.

In some embodiments, step 208 may further include sub-steps 2082-2088.

Sub-step 2082, a legal image of a preset size is obtained. The legalimage includes the target feature area.

In this step, in the portrait processing scene, the legal image with thepreset size may be a standard face front view with a preset size; andthrough a face recognition technology, the face area of the standardface front view may be determined as the target feature area.

Sub-step 2083, the legal key point is extracted from the target featurearea of the legal image.

In this step, in the portrait processing scene, through the facedetection technology, the legal key point may be extracted from the facearea of the standard face front view, and the legal key point may be apoint on the facial feature outline.

Sub-step 2084, the mask corresponding to the video frame image is mappedto an empty graph layer with a preset size according to the topologicalrelation information, the mask and the legal key point.

In this step, the topological relation information may include a griddiagram constructed by the key points according to relative positionvectors among the key points, where a triangular grid is formed amongthe key points, by combining the topological relation information andthe legal key point, the mask corresponding to the video frame image maybe mapped to the empty graph layer with the preset size in a legal way,and all rendering results of rendering operation on the special effectmap in the special effect video frame before the current time may bereflected by channel information of the empty graph layer.

Sub-step 2085, after transparency to the empty graph layer is set to apreset degree, the weight value of the special effect video frame isobtained according to the channel information of the empty graph layer.

In this step, after the transparency of the empty graph layer is set tothe preset degree, the empty graph layer is changed from a previous rgb(red channel, green channel, blue channel) format to an rgba (redchannel, green channel, blue channel, transparent channel) format, sothat the weight value of the special effect video frame may be obtainedaccording to the channel information of the empty graph layer, where adegree value of the transparency may be determined by the graphicdesigner.

For example, assuming that a user determines an eye makeup specialeffect map A and a lip makeup special effect map B, the user may renderthe eye makeup special effect map A to a graph layer a of the videoframe in the video frame of the video, and render the lip makeup specialeffect map B to a graph layer b of the video frame, so as to obtain aspecial effect video frame (A+a) and a special effect video frame (B+b).

Mixing the special effect video frame (A+a) and the special effect videoframe (B+b) by using the channel information of the empty graph layer toobtain a corresponding replacement video frame includes: setting a pixelcolor on the special effect video frame (A+a) as ColorCA (u, v), where uand the v are texture coordinates. A pixel color on the special effectvideo frame (B+b) is ColorCB (u, v). A color of the transparent channelin the empty graph layer is alpha (u,v), then a color of each pixel inthe replacement video frame is ColorC (u,v), ColorC (u,v)=ColorCA(u,v)×Alpha (u,v)+ColorCB (u,v)×(1−Alpha (u, v)), wherein the Alpha(u,v) may serve as a weight value of the special effect video frame(A+a), and the (1−Alpha (u, v)) may serve as a weight value of thespecial effect video frame (B+b).

In sub-step 2086, the pixel value of each pixel in the empty graph layeris extracted.

In some embodiments, sub-step 2086 may further include sub-steps A1 andA2.

Sub-step A1, the empty graph layer is sent to a central processing unitthrough a graphics processor.

In the embodiments of the present disclosure, other operations beforesub-step 2086 are implemented through the graphics processor. When pixelvalue extraction of each pixel in the empty graph layer is started, theempty graph layer may be sent to the central processing unit through thegraphics processor, and the central processing unit realizes the pixelvalue extraction of each pixel in the empty graph layer, which mayimprove extraction efficiency.

Sub-step A2, the pixel value of each pixel in the empty graph layer isextracted through the central processing unit.

In this step, the central processing unit realizes the pixel valueextraction of each pixel in the empty graph layer, which may improve theextraction efficiency.

In sub-step 2087, a swipe area value is determined according to thepixel value corresponding to each pixel.

In this step, determining the swipe area value according to the pixelvalue corresponding to each pixel includes: from all the pixels, targetpixels whose pixel values are greater than a preset pixel value areselected, the pixel values of the target pixels are summed up, and aresult of sum is divided by the number of the target pixels to obtainthe swipe area value.

In sub-step 2088, if the swipe area value is greater than or equal to apreset threshold value, the special effect map in the special effectvideo frame is replaced with another special effect map.

In the embodiments of the present disclosure, a swipe area may beconfigured to reflect a degree to which the target feature area of thespecial effect video frame is affected by the shielded area. Therefore,according to the swipe area value, when to perform the next specialeffect map correction may be judged. Specifically, after thecorresponding transparency of the empty graph layer is set, the emptygraph layer records rendering results of all the rendering operations onthe special effect map in the special effect video frame before thecurrent moment, and when the swipe area value is greater than or equalto the preset threshold value, the special effect map in the specialeffect video frame may be replaced with another special effect map.

For example, if there are the two special effect maps A and B, as wellas the special effect video frame (A+a) and the special effect videoframe (B+b) corresponding to the two special effect maps A and B, whenthe swipe area value is greater than or equal to the preset thresholdvalue, the special effect maps A and B in the special effect video frame(A+a) and the special effect video frame (B+b) may be exchanged. Thepurpose of exchanging respective weight values of the special effectvideo frame (A+a) and the special effect video frame (B+b) is achievedand reflected in the video picture, namely, the user may achieve aneffect of exchanging the special effect maps A and B in the picturethrough influences of the shielded area on the target feature area.

In addition, the special effect maps A and B in the special effect videoframe (A+a) and the special effect video frame (B+b) may further bereplaced with new special effect maps C and D to obtain a special effectvideo frame (C+a) and a special effect video frame (D+b), and respectiveweight values of the special effect video frame (A+a) and the specialeffect video frame (B+b) are assigned to the corresponding specialeffect video frame (C+a) and the corresponding special effect videoframe (D+b). It is reflected in the video picture, namely, the user mayachieve the purpose of replacing the special effect maps A and B in thepicture with the other special effect maps C and D through theinfluences of the shielded area on the target feature area.

Step 209, the corresponding replacement video frame is generatedaccording to the special effect video frame and the weight value.

Implementation of this step is similar to implementation of step 104above, so the implementation of this step will not be described indetail in the embodiments of the present disclosure.

Step 210, the corresponding special effect video frame in the video isreplaced with the replacement video frame.

Implementation of this step is similar to implementation of step 105above, so the implementation of this step will not be described indetail in the embodiments of the present disclosure.

In summary, by means of another method for processing a video providedby the embodiments of the present disclosure, the special effect videoframe set with the preset special effect map in the video may beobtained. The mask corresponding to the target feature area is extractedfrom the special effect video frame. If the shielded area exists in thetarget feature area of the mask, the weight value of the special effectvideo frame is determined according to the topological relationinformation corresponding to the special effect video frame, the maskand the preset legal key point. According to the special effect videoframe and the weight value, the corresponding replacement video frame isgenerated. The replacement video frame replaces the correspondingspecial effect video frame in the video. In the present disclosure,through the mask with the shielded area, the topological relationinformation corresponding to the special effect video frame and thelegal key point, the weight value of a correction degree of the specialeffect map is determined. When shooting the video, the user may shieldan area with the special effect map through a body or other objects toachieve real-time special effect display intensity adjustment of thespecial effect map in the area, and during an entire adjustment process,the user only needs to shield a picture instead of setting relatedparameters, so that a process of the special effect display intensityadjustment is simplified, and smoothness of the special effect displayintensity adjustment is improved.

FIG. 4 is a block diagram of a device for processing a video provided bythe embodiments of the present disclosure. As shown in FIG. 4, thedevice includes a special effect obtaining module 401, a firstextraction module 402 and a weight module 403.

The special effect obtaining module 401 is configured to obtain aspecial effect video frame set with a preset special effect map in avideo.

The first extraction module 402 is configured to extract a maskcorresponding to a target feature area from the special effect videoframe.

The weight module 403 is configured to determine a weight value of thespecial effect video frame according to topological relation informationcorresponding to the special effect video frame, the mask and a presetlegal key point if a shielded area exists in the target feature area ofthe mask.

In some embodiments, the weight module 403 includes a mappingsub-module, a weight sub-module, an obtaining sub-module, a firstextraction sub-module and a second extraction sub-module.

a mapping sub-module is configured to map the mask corresponding to avideo frame image to an empty graph layer with a preset size accordingto the topological relation information, the mask and the legal keypoint.

The weight sub-module is configured to obtain the weight value of thespecial effect video frame according to channel information of the emptygraph layer after transparency to the empty graph layer is set to apreset degree.

The obtaining sub-module configured to obtain a legal image of a presetsize, wherein the legal image includes the target feature area.

The first extraction sub-module configured to extract the legal keypoint in the target feature area of the legal image.

The second extraction sub-module is configured to extract a pixel valueof each pixel in the empty graph layer.

In some embodiments, the second extraction sub-module is furtherconfigured to: send the empty graph layer to a central processing unitthrough a graphics processor, and extract the pixel value of each pixelin the empty graph layer through the central processing unit.

A determination sub-module is configured to determine a swipe area valueaccording to the pixel value corresponding to each pixel.

A replacement sub-module is configured to replace the special effect mapin the special effect video frame with another special effect map if theswipe area value is greater than or equal to a preset threshold value.

Optionally, the weight module 403 is further configured to: determinethe weight value of the special effect video frame according to thetopological relation information corresponding to the special effectvideo frame, the mask and the preset legal key point if a non-mask areaexists in the target feature area of the mask.

A generating module 404 is configured to generate a correspondingreplacement video frame according to the special effect video frame andthe weight value.

A replacement module 405 is configured to replace the correspondingspecial effect video frame in the video with the replacement videoframe.

In some embodiments, the device further includes the following modules:a second extraction module configured to extract a plurality of videoframes in the video according to a preset time period; a collectionmodule configured to collect a plurality of target key points of thetarget feature area in the video frame; a position determination moduleconfigured to take relative position vector information among theplurality of target key points as the topological relation information;an enlargement module configured to increase a size of the maskaccording to a preset multiple; and a fuzzy module configured to performfuzzy treatment on the mask.

In summary, by means of the video processing apparatus provided by theembodiments of the present disclosure, the special effect video frameset with the preset special effect map in the video may be obtained. Themask corresponding to the target feature area is extracted from thespecial effect video frame. If the shielded area exists in the targetfeature area of the mask, the weight value of the special effect videoframe is determined according to the topological relation informationcorresponding to the special effect video frame, the mask and the presetlegal key point. According to the special effect video frame and theweight value, the corresponding replacement video frame is generated.The replacement video frame replaces the corresponding special effectvideo frame in the video. In the present disclosure, through the maskwith the shielded area, the topological relation informationcorresponding to the special effect video frame and the legal key point,the weight value of a correction degree of the special effect map isdetermined. When shooting the video, the user may shield an area withthe special effect map through a body or other objects to achievereal-time special effect display intensity adjustment of the specialeffect map in the area, and during an entire adjustment process, theuser only needs to shield a picture instead of setting relatedparameters, so that a process of the special effect display intensityadjustment is simplified, and smoothness of the special effect displayintensity adjustment is improved.

FIG. 5 is a block diagram of an electronic device 600 according to anexemplary embodiment. For example, the electronic device 600 may be amobile phone, a computer, a digital broadcasting terminal, messagingdevice, a game console, tablet device, medical device, fitness device, apersonal digital assistant or the like.

Referring to FIG. 5, the electronic device 600 may include one or moreof the following components: a processing component 602, a memory 604, apower component 606, a multimedia component 608, an audio component 610,an input/output (I/O) interface 612, a sensor component 614 and acommunication component 616.

The processing component 602 generally controls overall operation of theelectronic device 600, such as operation associated with displaying,telephone calling, data communication, camera operation, and recordingoperation. The processing component 602 may include one or moreprocessors 620 to execute an instruction to complete all or part ofsteps in the above method.

In addition, the processing component 602 may include one or moremodules to facilitate interaction between the processing component 602and other components. For example, the processing component 602 mayinclude a multimedia module to facilitate interaction between themultimedia component 608 and the processing component 602.

The memory 604 is configured to store various types of data to supportthe operation on the electronic device 600. Examples of the data includethe instruction for any application or method operated on the electronicdevice 600, contact data, phone book data, a message, a picture, avideo, and the like. The memory 604 may be achieved by any type ofvolatile or non-volatile storage device or a combination thereof, suchas a static random access memory (SRAM), an electrically erasableprogrammable read only memory (EEPROM), an erasable programmable readonly memory (EPROM), a programmable read only memory (PROM), a read onlymemory (ROM), a magnetic memory, a flash memory, a magnetic disk or anoptical disk.

The power component 606 provides power to various components of theelectronic device 600. The power component 606 may include a powermanagement system, one or more power sources, and other componentsassociated with generating, managing, and distributing power of theelectronic device 600.

The multimedia component 608 includes a screen that provides an outputinterface between the electronic device 600 and a user. In someembodiments, the screen may include a liquid crystal display (LCD) and atouch panel (TP). If the screen includes the touch panel, the screen maybe implemented as a touch screen to receive an input signal from theuser. The touch panel includes one or more touch sensors to sense touch,swipe, and gestures on the touch panel. The touch sensor may not onlysense a boundary of a touch or swipe action, but also detect durationand pressure related to touch or swipe operation. In some embodiments,the multimedia component 608 includes a front camera and/or a rearcamera. When the electronic device 600 is in an operation mode, such asa shooting mode or a video mode, the front camera and/or the rear cameramay receive external multimedia data. Each of the front camera and therear camera may be a fixed optical lens system or have focal length andoptical zoom capability.

The audio component 610 is configured to output and/or input an audiosignal. For example, the audio component 610 includes a microphone(MIC). When the electronic device 600 is in the operation mode, such asa call mode, a recording mode, and a voice recognition mode, themicrophone is configured to receive the external audio signal. Thereceived audio signal may be further stored in the memory 604 ortransmitted via the communication component 616. In some embodiments,the audio component 610 further includes a speaker configured to outputthe audio signal.

The I/O interface 612 provides an interface between the processingcomponent 602 and a peripheral interface module. The peripheralinterface module may be a keyboard, a click wheel, a button or the like.The buttons may include but are not limited to a home button, a volumebutton, a start button, and a lock button.

The sensor component 614 includes one or more sensors configured toprovide status assessment of various aspects for the electronic device600. For example, the sensor component 614 may detect an on/off state ofthe electronic device 600, and relative positioning of the components,for example, the components are a display and a keypad of the electronicdevice 600, and the sensor component 614 may further detect positionchange of the electronic device 600 or one of the components of theelectronic device 600, presence or absence of contact between the userand the electronic device 600, orientation or acceleration/decelerationof the electronic device 600, and temperature change of the electronicdevice 600. The sensor component 614 may include a proximity sensorconfigured to detect presence of a nearby object without any physicalcontact. The sensor component 614 may further include an optical sensor,such as a CMOS or CCD image sensor, configured to being used in animaging application. In some embodiments, the sensor component 614 mayfurther include an acceleration sensor, a gyro sensor, a magneticsensor, a pressure sensor, or a temperature sensor.

The communication component 616 is configured to facilitate wired orwireless communication between the electronic device 600 and anotherdevice. The electronic device 600 may access a wireless network based ona communication standard, such as WiFi, an operator network (such as 2G,3G, 4G, or 5G), or a combination thereof. In the exemplary embodiment,the communication component 616 receives a broadcast signal or broadcastrelated information from an external broadcast management system via abroadcast channel In the exemplary embodiment, the communicationcomponent 616 further includes a near field communication (NFC) moduleto facilitate short-range communication. For example, the NFC module maybe implemented based on a radio frequency identification (RFID)technology, an infrared data association (IrDA) technology, anultra-wideband (UWB) technology, a bluetooth (BT) technology and othertechnologies.

In some embodiments, the electronic device 600 may be implemented by oneor more application specific integrated circuits (ASIC), digital signalprocessors (DSP), digital signal processing devices (DSPD), programmablelogic devices (PLD), field programmable gate array (FPGA), controllers,microcontrollers, microprocessors or other electronic components to beconfigured to obtain a special effect video frame set with a presetspecial effect map set in a video. In the special effect video frame, amask corresponding to a target feature area is extracted. If a shieldedarea exists in the target feature area of the mask, a weight value ofthe special effect video frame is determined according to topologicalrelation information corresponding to the special effect video frame,the mask and a preset legal key point. A corresponding replacement videoframe is generated according to the special effect video frame and theweight value. The corresponding special effect video frame in the videois replaced with the replacement video frame.

In some embodiments, a non-transitory storage medium including theinstruction, such as the memory 604 including the instruction, isfurther provided. The above instruction may be executed by the processor620 of the electronic device 600 to complete the above method. Forexample, the non-transitory storage medium may be the ROM, a randomaccess memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, opticaldata storage device or the like.

FIG. 6 is a block diagram of an electronic device 700 according to anexemplary embodiment. For example, the electronic device 700 may beprovided as a server. Referring to FIG. 6, the electronic device 700includes a processing component 722 further including one or moreprocessors, and memory resources represented by a memory 732 configuredto store an instruction capable of being executed by the processingcomponent 722, such as an application program. The application programstored in the memory 732 may include one or more modules, wherein eachof the modules corresponds to the set of instructions. In addition, theprocessing component 722 is configured to execute the instruction toperform acquiring a special effect video frame set with a preset specialeffect map in a video. A mask corresponding to a target feature area isextracted from the special effect video frame. If a shielded area existsin the target feature area of the mask, a weight value of the specialeffect video frame is determined according to topological relationinformation corresponding to the special effect video frame, the maskand a preset legal key point. According to the special effect videoframe and the weight value, a corresponding replacement video frame isgenerated. The replacement video frame replaces the correspondingspecial effect video frame in the video.

The electronic device 700 may further include a power component 726configured to perform power management of the electronic device 700, awired or wireless network interface 750 configured to connect theelectronic device 700 to a network, and an input/output (I/O) interface758. The electronic device 700 may operate an operating system based onbeing stored in the memory 732, such as Windows Server™, Mac OS X™,Unix™, Linux™, FreeBSD™, or the like.

The embodiments of the present disclosure further provide an applicationprogram. When the application program is executed by a processor ofelectronic device, a method provided by the present disclosure isimplemented, wherein the method includes steps that the special effectvideo frame set with the preset special effect map in the video may beobtained, the mask corresponding to the target feature area is extractedfrom the special effect video frame, if the shielded area exists in thetarget feature area of the mask, the weight value of the special effectvideo frame is determined according to the topological relationinformation corresponding to the special effect video frame, the maskand the preset legal key point, according to the special effect videoframe and the weight value, the corresponding replacement video frame isgenerated, and the replacement video frame replaces the correspondingspecial effect video frame in the video.

Other embodiments of the present disclosure will be apparent to thoseskilled in the art from consideration of the specification and practiceof the disclosure herein. The disclosure is intended to cover anyvariations, uses, or adaptations of the present disclosure following thegeneral principles thereof and including such departures from thedisclosure as come within known or customary practice in the art. It isintended that the specification and embodiments be considered asexemplary only, with a true scope and spirit of the present disclosurebeing indicated by the following claims.

It will be appreciated that the present disclosure is not limited to theexact construction that has been described above and illustrated in theaccompanying drawings, and that various modifications and changes may bemade without departing from the scope thereof. It is intended that thescope of the present disclosure only be limited by the appended claims.

What is claimed is:
 1. A device for processing a video, comprising amemory and at least one processor, wherein the at least one processor isconfigured to read and execute instructions stored in the memory to:determine a special effect video frame of the video, wherein a targetfeature area of the special effect video frame comprises a presetspecial effect map; extract a mask corresponding to the special effectvideo frame; determine a weight value of the special effect video framebased on topological relation information corresponding to the specialeffect video frame, the mask and a preset legal key point upondetermining that the shielded area exists in the target feature area ofthe mask; generate a replacing video frame based on the special effectvideo frame and the weight value; and replace the special effect videoframe with the replacing video frame.
 2. The device according to claim1, wherein the at least one processor is further configured to read andexecute instructions stored in the memory to: map the mask to an emptygraph layer based on the topological relation information, the mask, andthe preset legal key point, wherein the empty graph layer has a presetsize; and obtain the weight value based on channel information of theempty graph layer, after setting transparency of the empty graph layerto a preset degree.
 3. The device according to claim 2, wherein the atleast one processor is further configured to read and executeinstructions stored in the memory to: obtain a legal image with thepreset size, wherein the legal image comprises the target feature area;and extract the preset legal key point from the target feature area. 4.The device according to claim 2, wherein the at least one processor isfurther configured to read and execute instructions stored in the memoryto: extract a pixel value of each pixel in the empty graph layer;determine a swipe area value according to the pixel value of each pixel;and replace the special effect map with another special effect map inresponse to the swipe area value being greater than or equal to a firstpreset threshold value.
 5. The device according to claim 4, wherein theat least one processor is further configured to read and executeinstructions stored in the memory to: determine a plurality of targetpixels whose pixel values are greater than or equal to a second presetthreshold value from all the pixels of the empty graph layer; and take aratio of a sum of the pixel values of the target pixels to the number ofthe target pixels as the swipe area value.
 6. The device according toclaim 4, wherein the at least one processor is further configured toread and execute instructions stored in the memory to: send the emptygraph layer to a central processing unit through a graphics processor;and extract the pixel value of each pixel in the empty graph layer bythe central processing unit.
 7. The device according to claim 1, whereinthe at least one processor is further configured to read and executeinstructions stored in the memory to: determine the weight value of thespecial effect video frame based on the topological relationinformation, the mask and the preset legal key point upon determiningthat a non-mask area exists in the target feature area of the mask. 8.The device according to claim 1, wherein the at least one processor isfurther configured to read and execute instructions stored in the memoryto: extract a plurality of video frames of the video based on a presettime period; collect a plurality of target key points of target featureareas in the video frames; and determine the topological relationinformation based on relative position vector information among theplurality of target key points.
 9. The device according to claim 1,wherein the at least one processor is further configured to read andexecute instructions stored in the memory to: increase a size of themask based on a preset multiple; and fuzz the mask.
 10. A non-transitorycomputer readable storage medium, wherein the computer readable storagemedium is stored with instructions thereon, and the instructions areexecuted by a processor to: determine a special effect video frame ofthe video, wherein a target feature area of the special effect videoframe comprises a preset special effect map; extract a maskcorresponding to the special effect video frame; determine a weightvalue of the special effect video frame based on topological relationinformation corresponding to the special effect video frame, the maskand a preset legal key point upon determining that the shielded areaexists in the target feature area of the mask; generate a replacingvideo frame based on the special effect video frame and the weightvalue; and replace the special effect video frame with the replacingvideo frame.
 11. The non-transitory computer readable storage mediumaccording to claim 10, wherein the instructions are executed by theprocessor to: map the mask to an empty graph layer based on thetopological relation information, the mask, and the preset legal keypoint, wherein the empty graph layer has a preset size; and obtain theweight value based on channel information of the empty graph layer,after setting transparency of the empty graph layer to a preset degree.12. The non-transitory computer readable storage medium according toclaim 11, wherein the instructions are executed by the processor to:obtain a legal image with the preset size, wherein the legal imagecomprises the target feature area; and extract the preset legal keypoint from the target feature area.
 13. The non-transitory computerreadable storage medium according to claim 11, wherein the instructionsare executed by the processor to: extract a pixel value of each pixel inthe empty graph layer; determine a swipe area value according to thepixel value of each pixel; and replace the special effect map withanother special effect map in response to the swipe area value beinggreater than or equal to a first preset threshold value.
 14. Thenon-transitory computer readable storage medium according to claim 13,wherein the instructions are executed by the processor to: determine aplurality of target pixels whose pixel values are greater than or equalto a second preset threshold value from all the pixels of the emptygraph layer; and take a ratio of a sum of the pixel values of the targetpixels to the number of the target pixels as the swipe area value. 15.The non-transitory computer readable storage medium according to claim13, wherein the instructions are executed by the processor to: send theempty graph layer to a central processing unit through a graphicsprocessor; and extract the pixel value of each pixel in the empty graphlayer by the central processing unit.
 16. The non-transitory computerreadable storage medium according to claim 10, wherein the instructionsare executed by the processor to: determine the weight value of thespecial effect video frame based on the topological relationinformation, the mask and the preset legal key point upon determiningthat a non-mask area exists in the target feature area of the mask. 17.The non-transitory computer readable storage medium according to claim10, wherein the instructions are executed by the processor to: extract aplurality of video frames of the video based on a preset time period;collect a plurality of target key points of target feature areas in thevideo frames; and determine the topological relation information basedon relative position vector information among the plurality of targetkey points.
 18. The non-transitory computer readable storage mediumaccording to claim 10, wherein the instructions are executed by theprocessor to: increase a size of the mask based on a preset multiple;and fuzz the mask.